Systems And Methods For Presenting Location Related Information

ABSTRACT

Systems and methods for presenting location related information after a user arrives at a place. In an aspect, when a user gazes at a display of standby device, information presentation begins. In another aspect, when a user shakes a standby device and then gazes at it, a presentation starts. Location related info may be sorted and presented by the pointing direction of device.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 U.S.C. Sec. 119 ofprovisional patent application Ser. No. 61/896,632, filed Oct. 28, 2013and Ser. No. 62/068,754, filed Oct. 26, 2014.

FEDERALLY SPONSORED RESEARCH

Not applicable

SEQUENCE LISTING OR PROGRAM

Not applicable

BACKGROUND

1. Field of Invention

This invention relates to presenting information, more particularly topresenting location related information utilizing gaze detection.

2. Description of Prior Art

Many portable electronic devices have become ubiquitous, as anindispensible part of our daily life. Examples include smartphones,tablet computers, and potentially, smart watches or other wearablegadgets. These devices, especially smartphones, may be used to transmitto users and then present information such as advertisement prepared forconsumers, notice and info for event attendees, class messages forstudents, or flight info for passengers. But many a time, it is not easyto acquire contact info of people involved and to figure out when topresent. For instance, most ads are delivered to peopleindiscriminately, blindly, and without specific consideration on timing,which compromises the ad effect severely.

To make ad more relevant and effective, location-based advertising hasbeen advocated. For instance, people visiting a store have a betterchance to become a customer than people elsewhere. So a store managermay be more interested in sending ads to people present at the storethan people at home. The same is true for delivery of information otherthan advertisement. For example, event attendees are more willing toread event material when they are in there, students are more likely toread class message when at school, and passengers are more eager tolearn flight and gate status when at the airport. Moreover, it'srelatively straightforward to send location related information, sincedevices on the scene are the obvious target, and it may start rightafter users arrive at a location or come near a location. As a result,it's likely that right info is sent to the right people in the rightplace at the right time. But then, the next issue may be how to presentit in such a way that it is easy, simple, and convenient for user toaccess. If info is transmitted via email, a method used quite oftennowadays, people may have to go through several steps to log in emailaccount, open a mail, and then take a look at it. If viewing inforequires an app, people have to find the app among other apps installedin a device and then launch it. Either way, it is not convenient enoughto look for info transmitted from the network. On the other hand, if adevice is on, and info content pops up by itself, it may becomeannoying; and if a device is in standby mode with a dark screen, it isinappropriate to lighten up the display to show any content without userconsent. Thus presenting info on a device automatically has its ownlimitations.

Therefore, there exists a need to present location related informationin a simple, easy, and convenient way.

OBJECTS AND ADVANTAGES

Accordingly, several main objects and advantages of the presentinvention are:

-   -   a). to provide improved methods and systems to present location        related information;    -   b). to provide such method and system which start presentation        when user gazes at it;    -   c). to provide such method and system which start presentation        when user shakes it and then gazes at it, with more user        control;    -   d). to provide such method and system which sort location        related information by device pointing direction; and    -   e). to provide such method and system which make info access        easy, simple, and convenient.

Further objects and advantages will become apparent from a considerationof the drawings and ensuing description.

SUMMARY

In accordance with the present invention, methods and systems areproposed to present location related information. After a user arrivesat a place, the user may just look at a device screen to start infopresentation by gaze. The user may also shake a device first to triggergaze detection, and then watch it to bring out a presentation by gaze.The method makes it easy and convenient for users to view locationrelated information. Moreover, location related information may besorted and presented by device pointing direction for simplifiedpresentation and easy understanding.

DRAWING FIGURES

FIG. 1 is an exemplary block diagram describing one embodiment inaccordance with the present invention.

FIG. 2 is an exemplary flow diagram showing one embodiment of presentinglocation related information in accordance with the present invention.

FIG. 3 is an exemplary flow diagram showing another embodiment ofpresenting location-based information in accordance with the presentinvention.

FIG. 4 uses graphic diagrams to show yet another embodiment ofinformation presentation in accordance with the present invention.

REFERENCE NUMERALS IN DRAWINGS

10 Sensor 12 Device 14 Processor 16 Computer Readable Medium 18 Sensor20 Sensor 22 Sensor 36 Eye 38 Smartphone 100, 102, 104, 106, 108, 110,112, 114, 116, 118, 120, 122, 124, and 126 are exemplary steps.

DETAILED DESCRIPTION

FIG. 1 is an illustrative block diagram of one embodiment according tothe present invention. A device 12 may represent an electronic device,including but not limited to mobile phone, smart phone, smart watch,wearable device, handheld computer, tablet computer, and the like.Device 12 may include a processor 14 and computer readable medium 16.Processor 14 may mean one or more processor chips or systems. Medium 16may include a memory hierarchy built by one or more memory chips orstorage modules like RAM, ROM, FLASH, magnetic, optical and/or thermalstorage devices. Processor 14 may run programs or sets of executableinstructions stored in medium 16 for performing various functions andtasks, e.g., surfing on the Internet, playing video or music, gaming,electronic payment, social networking, sending and receiving emails,messages, files, and data, executing other applications, etc. Device 12may also include input, output, and communication components, which maybe individual modules or integrated with processor 14. The communicationcomponents may connect the device to another device or a communicationnetwork. Usually, Device 12 may have a display (not shown in FIG. 1 forbrevity reason) and a graphical user interface (GUI). A display may haveliquid crystal display (LCD) screen, organic light emitting diode (OLED)screen (including active matrix OLED (AMOLED) screen), or LED screen. Ascreen surface may be sensitive to touches, i.e., sensitive to hapticand/or tactile contact with a user, especially in the case of smartphone, smart watch, and tablet computer. A touch screen may be used as aconvenient tool for user to enter input and interact with a system.Furthermore, device 12 may also have a voice recognition component forreceiving verbal command or audio input from a user.

A communication network which device 12 may be connected to may cover arange of entities such as the Internet or the World Wide Web, a localarea network (LAN), a wide area network (WAN), a metropolitan areanetwork (MAN), a telephone network, an intranet, wireless, and othertypes of networks. Device 12 may be connected to a network by variouswired, wireless, optical, infrared, ultrasonic or other communicationmeans.

Device 12 may also include a sensor 10 which tracks the eye movement orgazing direction of user using mature eye-tracking or gaze detectiontechnologies. The sensor may be arranged on the top surface of device,or close to a display screen, and may be designed to have imagingcapability. With imaging functions, a system may recognize whetherusers' eye is in such a position that the eye sight falls on the body ofdevice 12 using certain algorithm, in other words, sensor 10 may beemployed to determine whether a user is looking at the body or thescreen of a device. Once it senses that a user is gazing or looking at agiven target, it may record the starting time, and then the total gazingor watching time. Only when a gazing or watching time exceeds certainvalue, for instance a few seconds, it may be declared that a user isgazing or looking at a target. As a consequence, a very brief look maybe too short to qualify as gazing or watching act. In the followingsections, it is assumed the total time satisfies the minimum valuerequirement when it is said gazing or looking is detected.

Sensor 10 may be built using mature imaging technologies, such as cameramodules used in almost every smartphone, and the image of user's eye maybe analyzed with mature algorithm to decide which direction a user islooking at. Both visible and infrared light may be employed for eyetracking. In the latter case, infrared light source may be arranged toprovide a probing beam. In addition, sensor 10 may also employ othersuitable technologies which are capable and affordable other than theeye-analysis scheme discussed to determine the gazing or watchingdirection of user. In some applications, when the accuracy of gazingdirection is not critical, such as when a gaze target is a screen, not asmall area of the screen, a watching direction may be obtained viaanalyzing the facial picture of user.

Device 12 may also include a sensor 20 which functions as proximitydetector, which is well known in the art and well developed too. Sensor20 may be used to detector an object outside of the device and may havemultiple sensing units. It may include a camera-like system to obtainimages as well as infrared images and then recognize any movementthrough image analysis over a period of time. It may also havecapability to sense whether device 12 is close to the body of user orwhether is held by a hand. The detection result may be used to determinethe environment where a user is in, or the intention of a user. Forinstance, a user may want to look at a device anytime when he is holdingit on hand.

Moreover, device 12 may contain a sensor 18 to detect its own movementby sensing acceleration, deceleration, and rotation, which may bemeasured by accelerometers and gyroscopes. Accelerometers and gyroscopesare already mass produced using semiconductor technologies. They arewidely used in smartphones and other personal gadgets. Using dataobtained by sensor 18, it can be determined whether device 12 is movedto the left, right, forward, or backwards, and at what speed, whether itis rotated clockwise or anticlockwise along which axis, and whether itis tilted to the left, right, forward, or backwards. The data may alsobe used to detect whether a device is moved back and forth as a resultof shaking. In some embodiments in the following, device shaking, as auser input, is one state to be detected. When a user causes a device incertain motion as a way to give instructions to it, the word “shake” or“shaking” may be interpreted here as many forms of movement such asmoving horizontally or vertically, rotating along any axis, or any otherpatterns of back and forth movement. Furthermore, sensor 18 may be usedto detect vibration of device 12. Thus, knocking or tapping on devicebody may be utilized too, because it generates detectable vibrationsignals.

Inside device 12, output signals of sensors and detectors aretransmitted to processor 14, which, employed with certain algorithm, mayprocess the data and produce subsequent command instructions accordingto certain programs or applications. The instructions may includepresenting location related info on a screen.

In addition, device 12 may carry a positioning sensor (not shown in FIG.1 for brevity) and a magnetic sensor 22 as electronic compass. Apositioning sensor may be a global positioning system (GPS), whichenables a device to get its own location info. Device position may alsobe obtained using wireless triangulation methods, or a system usingother suitable technologies, while both may be performed by a serviceprovider or service facility. Sensor 22 measures the earth magneticfield along at least two orthogonal axes X and Y. It may be used todetermine device orientation, such as which direction a device ispointing at, assuming the device is placed in a horizontal or verticalposition. When a device's location is known, a service center may sendto the device location-based information, i.e., info related to thelocation or nearby places. In the case of location-based advertising, auser may receive commercials after he or she is at a business or closeto a business. On the other hand, when the pointing direction of deviceis known, space around a user may be divided into sections. With theknowledge of device's location and pointing direction, a segment of maparea which corresponds to where a device is pointing at may begenerated. The segment may match user's interest, and thus informationfrom this segment may be more relevant than info from other areas.Meanwhile, sorting by segment may make information easier to view forusers, since the quantity of contents is reduced.

FIG. 2 is a schematic flow diagram showing one embodiment of presentinglocation related information. Take smartphone for example. Assume asmartphone is in standby mode in Step 100. When a user with the phoneenters Location A, a system sensor may detect it in Step 102. Forinstance, when a phone arrives at a place, a service provider may senseit or a local sensor may detect it using mature positioningtechnologies. Assume there is information available which is related toLocation A. In Step 104, location-based signal is transmitted to thephone and the phone receives it. The signal may come from a remotecenter or a nearby facility. Once the phone gets the signal, it startssensing user's gaze direction. When not triggered, gaze detectionfunction may be in off state to conserve power. In Step 106, the usergazes at the phone screen, which may be sensed by a gaze sensor likesensor 10 of FIG. 1. Here user's gaze act may work as user's approvalfor presenting information. In Step 108, the phone displays contentsrelated to Location A.

After arriving at a location, a user may become more likely to viewinformation related to the place. The user just needs to look at phonescreen, info would appear automatically. The info presentation processis easy, simple and convenient. It may be used by teacher to distributeclass notes, which may be accessed by students at one classroom only, bystore manager to send advertisement to people at or close to his or herstore only, or by organizer to send on-site event participants info onthe event. Usually for indoor or some urban environment, positioningmethods other than GPS are used, since GPS requires a clear view of thesky or clear line of sight for four GPS satellites.

The scheme described in FIG. 2 provides a simple and practical way toarrange location related information. But when lot of such informationis available, it makes things a little complicated. For instance, in ashopping mall area, there may be many stores and shops around. As aconsequence, a user may find it time consuming to get needed info. Thusa quick and convenient sorting method is desirable. For this issue,following discussion gives a solution.

FIG. 3 shows another schematic flow diagram of presenting locationrelated information. Assume a device is standby and is detected at aplace in Step 110. Next in Step 112, the device receives data whichcontains location related information through wireless technologies.Then, a gaze sensor is activated and begins to sense the gaze directionof user. The gaze sensor may be arranged always on if power conservationis not an issue and user consents. In Step 114, the gaze sensor detectswhether the user looks at the device. If the user looks elsewhere, thedevice may remain its standby state in Step 116. When the user ends thestandby state later on, a temporary icon may appear on screen. The iconmay represent information related to the location. Once tapped orclicked, location related info may be presented. A temporary icon mayalso be generated on screen for later use when a user is busy engagingwith an app at the moment of receiving location related information.Such icon provides another opportunity to present temporarily storedlocation related contents. Back to the figure, if it is detected thatthe user looks at the device for a given period of time, the device maystart to detect its orientation using a magnetometer component likesensor 22 of FIG. 1, as shown in Step 118. In the meantime, the devicemay acquire its position status, i.e., its location. The position infomay be obtained via the device's own sensor or received from an outsidesensing system. Once info of location and orientation is known, thedevice may start presentation of related information in Step 120. Therelated information is of info associated with the pointing direction ofthe device. For instance, with the knowledge of location and orientationand certain algorithm, a device may provide a list of business which arelocated from the place of device to somewhere far away along itspointing direction. The list of business may be in a text file or shownon a segment of map. Thus pointing act may be used as a sorting tool,and a device may be designed to show information related to or aroundthe pointing direction only. Besides business and organizationalentities, pointing direction of device may also be used to get info ofproducts. For instance, a user may point a device at a section of storeto get prearranged info of that area, such as coupons and list ofmerchandize on sale in that direction.

A device may be in a horizontal position, or vertical position. Takesmartphone for instance. If a phone is in horizontal position, with itsdisplay screen being horizontal and parallel to the ground, a pointingdirection is what its front end points outwards in a horizontal plane.For a phone in vertical position, a pointing direction is what its backpoints at or its camera points at, which is the opposite direction ofwhat its screen faces.

As orientation data may be obtained fast through an electronic compass,pointing may lead to real-time info scanning. In Step 122, orientationis measured again. If there is no change, contents on display may remainin Step 124. If there is a change, meaning the device is rotated topoint at a new direction, another set of contents may be presented inresponse in Step 126. For example, when a user rotates a smartphonehorizontally along vertical axis, it may work like scanning with aprobing beam. It may be designed such that during scanning, onlyinformation related to the business which is straight ahead will show upon the screen. Thus a user may slowly rotate a device, like asmartphone, to view info at each direction, or point a device at aselected business to access info of that business directly.

FIG. 4 uses graphic diagrams to show another embodiment of presentinglocation related information. Again, a smartphone is used in a retailsetting. It starts with Step 1 when a positioning sensor finds asmartphone 38 at store A. The phone is in standby mode and has a darkscreen. A service facility sends the phone a signal, and the phonereceives location related information. Unlike the previous embodiment,gaze sensor of the device is not triggered by the location-based signal,but by user's physical act like shaking or tapping the device. In Step2, the user shakes phone 38, which is picked up by the phoneimmediately, e.g., within seconds. Then the control system of phone 38,like processor 14 of FIG. 1, sends a signal to the gaze sensor. The gazesensor starts sensing the user to determine whether he or she looks atthe phone screen. If it is detected that user's eye 36 is on the phonescreen for a predetermined period of time in Step 3, the device maybegin presenting store advertisement and coupons in Step 4.

In above discussion, a user needs to do two things, shaking a phonelightly and watching the screen shortly, and then info will bedisplayed. The scheme brings several merits. A user may have morecontrol over what time to show location related information. It mayreduce the chance of showing unwanted info by accidental gaze at adevice. In addition, as shaking reflects user's desire for certaincontent, it may help satisfy a user and help content owner likemerchants in the meantime.

CONCLUSION, RAMIFICATIONS, AND SCOPE

Thus it can be seen that systems and methods are introduced to presentlocation related information.

The improved method and system have the following features andadvantages:

-   -   (1). Location related info may be brought out by simply gazing        at screen;    -   (2). Gaze sensor of device may be turned on by a location-based        signal or shaking act of user;    -   (3). Information may be sorted and presented by the pointing        direction of device.

Although the description above contains many specificities, these shouldnot be construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodiments.Numerous modifications will be obvious to those skilled in the art.

Ramifications:

Ambient light sensor may be added to a device which may be used to senseambient light intensity to determine whether the device is in a pocketor bag. If a device is not pulled out, act of shaking, tapping orknocking may be ignored in applications discussed in the above.

Furthermore, a device may be equipped with facial recognition system.The system may at least recognize the device owner, which may protectuser privacy by not following other people's instructions. The systemmay make use of eye-tracking camera and employ facial sensing algorithmto identify a user.

In several examples in the above, shaking is used to illustrate variousembodiments. It is noted that other forms of user input, such asrotating, tilting, tapping, knocking, or touching may also be employedto perform the same function. Still other methods for user to interactwith a device besides shaking include sliding on a touch screen or touchpad, or opening a lid of device.

A user may speak to a device to turn on gaze sensor using voicerecognition techniques. For instance, a user may say to a device “Start”and then look at it to invoke a presentation.

If user's identity is known, info presented may be selected based on notonly the location of user, but also his or her past experience, or basedon user's experience only when it is worthwhile to do so. For instance,when a user is travelling, he or she may have more free time, whichprovides a good chance for presenting certain info, like commercials.Although advertisement presented may be unrelated to a location, sinceit may fit the interest or need of user, it may be relevant, and thuseffective.

The schemes illustrated in FIGS. 2, 3, and 4 may be combined in manyways. For example, sorting info by pointing function may be applied toall cases. Sorting function may be turned on or off by user any time.

For convenience and better user experience, after a user arrives at aplace and user's device receives location related data, the device maybeep to signal that some info is available to watch. Meanwhile, if thedevice is standby, a small window may appear to post a short notice.Thus, a user may easily know there is something available and then maygaze at the screen or notice window to invoke a presentation.

Lastly, for a qualified gaze or look at a display, user's eye may alsofall on things located outside of the display but close to its edge,instead of an area on display only. The reason is that, when a userlooks at objects close to a display, contents shown on it may also reachthe eye, thus providing a viewing opportunity anyway. And hopefully, theuser may turn his or her sight a bit to get a better reception of thecontents. Moreover in many cases, instead of display, it may be enoughto qualify as a gaze at display if a user just looks at the direction ofa device, because it means a user may have intention to watch apresentation on it, and the user may have a good chance to noticecontents displayed on the device anyway. In cases of smartphone andtablet computer, gazing at a device is almost equivalent to gazing at adisplay, because for these devices, a display may cover the whole areaof one side.

Therefore the scope of the invention should be determined by theappended claims and their legal equivalents, rather than by the examplesgiven.

1. A method performed for presenting information comprising: 1)providing an electronic device having a display and stored executableinstructions; 2) measuring an idling time period during which saiddevice remains idle or standby; 3) determining the location of saiddevice; 4) transmitting a location-based signal from a system to saiddevice; 5) sensing a user and determining whether said user looks at adirection toward said device, the sensing step getting started aftersaid device receives said signal; 6) measuring a watching time periodduring which said user looks at a direction toward said device, saidwatching time period overlapping a segment of said idling time period;7) presenting a plurality of contents using said display, said pluralityof contents arranged different from the presentation before said idlingtime period; and 8) said method arranged such that the presenting stepis performed when said idling time period and said watching time periodexceed a first and second predetermined value respectively.
 2. Themethod according to claim 1, further including detecting the orientationof said device, said plurality of contents arranged to be related tosaid location.
 3. The method according to claim 1 wherein said pluralityof contents is related to said location and/or said user.
 4. The methodaccording to claim 1, further including displaying a graphic object onsaid display when said idling time period is smaller than said firstpredetermined value or said watching time period is smaller than saidsecond predetermined value, said object when activated arranged to showsaid plurality of contents.
 5. The method according to claim 1 whereinsaid plurality of contents include at least one of the following items:news, advertisement, update, email, text message, tweet message, andinformation arranged for said user or by said user.
 6. A methodperformed for presenting information comprising: 1) providing anelectronic device having a display and stored executable instructions;2) measuring an idling time period during which said device remains idleor standby; 3) transmitting a location-based signal from a system tosaid device; 4) detecting a user input generated by a user involvingphysical movement of said user, said movement arranged to generate saidinput within a substantially short period of time; 5) sensing said userand determining whether said user looks at a direction toward saiddevice, the sensing step getting started after said user input isreceived; 6) measuring a watching time period during which said userlooks at a direction toward said device, said watching time periodoverlapping a segment of said idling time period; 7) presenting aplurality of contents using said display, said plurality of contentsarranged different from the presentation before said idling period; and8) said method arranged such that the presenting step is performed whensaid idling time period and said watching time period exceed a first andsecond predetermined value respectively.
 7. The method according toclaim 6, further including determining the location of said device. 8.The method according to claim 6 wherein said plurality of contents islocation related and/or user related.
 9. The method according to claim6, further including detecting the orientation of said device, saidplurality of contents arranged to be related to the pointing directionof said device.
 10. The method according to claim 6, further includingdisplaying a graphical object on said display when said idling timeperiod is smaller than said first predetermined value or said watchingtime period is smaller than said second predetermined value, said objectwhen activated arranged to show said plurality of contents.
 11. A systemhaving a display and stored executable instructions, comprising: 1)connection component for connecting said system to an electronic deviceand/or a communication network; 2) receiving means for receiving alocation-based signal transmitted from an facility; 3) detecting meansfor detecting a user input generated by a user involving physicalmovement of said user, said movement arranged to generate said inputwithin a substantially short period of time; 4) sensing means forsensing a user and determining whether said user looks at a directiontoward said display, the sensing activity via said sensing meanstriggered by said input; 5) first timing means for measuring an idlingtime period during which said system remains idle or standby; 6) secondtiming means for measuring a watching time period during which said userlooks at a direction toward said display, said watching time periodoverlapping a segment of said idling time period; 7) presentation meansfor presenting a plurality of contents using said display, saidplurality of contents arranged different from the presentation beforesaid idling period; and 8) said system configured such that said displaystarts showing said plurality of contents when said idling time periodand said watching time period go beyond a first and second predeterminedvalue respectively.
 12. The system according to claim 11, furtherincluding location means for supporting a sensor to determine thelocation of said system.
 13. The system according to claim 11 whereinsaid plurality of contents is location related and/or user related. 14.The system according to claim 11, further including orientation meansfor detecting the orientation of said system, said plurality of contentsarranged to be related to the pointing direction of said system.
 15. Thesystem according to claim 11 wherein said plurality of contents isarranged to change when the pointing direction of said system ischanged.